[CLI] META-248 — extract the DataHub adapter to workspacejson/datahub-agent - #10
Merged
Merged
Conversation
Steps 6 and 7 of META-248. The adapter was staged here only because META-240 had to preserve it somewhere while its permanent owner was decided. It is DataHub consumer logic, not neutral producer logic, and it now lives in workspacejson/datahub-agent as an internal module at src/adapters/workspacejson/. Removing the package is not a deletion — it had five dependent call sites: - packages/agents-audit-compat/src/package-metadata.test.ts asserted on the adapter's manifest. Repurposed to assert the extraction stayed done; it goes red if the directory returns. - scripts/check-architecture.test.mjs mutated the adapter's manifest in a red test. Replaced with two red tests that reject its RETURN, private or public. - scripts/check-architecture.mjs had two private-package-publication rules that became dead. Repurposed as repository-boundary rules. - .github/workflows/ci.yml gated on the adapter parity harness, which would have failed the build once the package was gone. - pnpm-lock.yaml, OWNERSHIP.md, README.md, CONTRIBUTING.md, AGENTS.md, CHANGELOG.md. migration/parity-datahub-shim.mjs travels with the adapter, as OWNERSHIP.md required. It runs in workspacejson/datahub-agent against the DataHub-owned candidate: 35/35, plus per-file source identity against this repository's own frozen pre-migration source. migration/parity-datahub-shim.txt stays as the dated record of what was verified before extraction. Step 7: neutral-producer-purity is unaffected — it scans packages/cli only — and still red-tests on DataHub, dbt and Vreko content. But it would NOT have caught a re-added sibling package, since that is outside packages/cli. That gap is closed by listing packages/datahub-adapter in FOREIGN_DIRECTORIES, so the boundary is enforced rather than merely documented. Guard: 20 red tests pass. packages/cli 35 tests, agents-audit 44 tests, build and typecheck clean. Also records, in OWNERSHIP.md, a defect found during extraction and still present here: types/ambient.d.ts shadows node:fs, which is what allowed the adapter's findDbtProjects to compile at all. Against real @types/node it does not. Same class of defect META-244 already fixed for @workspacejson/spec; the remaining node:fs shadowing is worth auditing on the same grounds.
![charliecreates[bot]](https://avatars.githubusercontent.com/in/1140780?s=60&v=4){kind=small}
There was a problem hiding this comment.
The extraction leaves a stale filesystem-path reference in AGENTS.md. The repository CI runs agents-audit scan . --fail-on error; its Node 22 job fails because the scanner treats ./packages/datahub-adapter/ as a missing-file reference.
Please describe the former location without a path-shaped token (or update the audit rule/configuration), then rerun the CI gate.
CI dogfoods agents-audit on this repository's own AGENTS.md, and its missing-file-reference rule resolves referenced paths. Naming the now-deleted packages/datahub-adapter/ directory failed that audit with exit 1 — the repository's own tool catching a stale doc reference. Describes the extraction without writing the dead path, and says why, so the next person does not helpfully add it back.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Steps 6 and 7 of META-248. Steps 3–5 (adoption, parity, integration test) landed in
workspacejson/datahub-agent.packages/datahub-adapter/was staged here only because META-240 had to preserve it somewhere while its permanent owner was decided.OWNERSHIP.mdalready said it was "staging, not architecture." It is DataHub consumer logic, and it now lives inworkspacejson/datahub-agentas an internal module atsrc/adapters/workspacejson/.This is not a deletion — there were five dependent call sites
Removing the directory alone would have broken the build. Each site was repaired, not deleted:
packages/agents-audit-compat/src/package-metadata.test.ts:57-65name,private,binscripts/check-architecture.test.mjs:121-126scripts/check-architecture.test.mjs:135-140repository-boundaryscripts/check-architecture.mjs:226,251private-package-publicationrules, now deadrepository-boundaryrules.github/workflows/ci.yml:102Plus
pnpm-lock.yaml,OWNERSHIP.md,README.md,CONTRIBUTING.md,AGENTS.md,CHANGELOG.md.The CI gate was the sharp edge
migration/parity-datahub-shim.mjsbecame a CI gate in #6, and itsnewside is the package being removed. Its own error message already anticipated this ("The DataHub adapter has been extracted from this repository (META-248)") but it stillprocess.exit(1), so CI would have gone red on merge.OWNERSHIP.mdsays the harness "travels with it and must pass against the DataHub-owned candidate after extraction" — so the harness is removed here and now runs there.migration/parity-datahub-shim.txtstays as the dated record of what was verified before extraction.Result at the DataHub-owned candidate: 35/35, plus per-file source identity against this repository's frozen pre-migration source (
workspace-json/agents-audit@e47eb1b8) — four of five files byte-identical, one documented type-only deviation. Provenance recorded indocs/provenance.mdthere.Step 7 — and a gap it did not cover
neutral-producer-purityis unaffected: it scanspackages/cli/for dbt/DataHub/Vreko content by filename and body, independent of the adapter's existence. It still red-tests on all three.But it would not have caught a re-added sibling package, because
packages/datahub-adapter/is outsidepackages/cli/. Listing it inFOREIGN_DIRECTORIEScloses that, using machinery already in the script:Three ways the return is now rejected, all red-tested: re-adding the directory; redefining the package name in any manifest (private or not —
private: trueis not a defence once the package is gone); a workflow referencing it for publication.Verification
The extraction guard was verified to actually bite — recreating
packages/datahub-adapter/turns both the metadata test and the architecture check red; removing it turns them green again.One defect found during extraction, still present here
Recorded in
OWNERSHIP.mdrather than fixed, since it is outside this PR's scope.The adapter typechecked in this repository only because
packages/datahub-adapter/tsconfig.jsonincludedtypes/ambient.d.ts, which shadowsnode:fswith a hand-writtenDirent. Against real@types/node@22.19.17,ReturnType<typeof readdirSync>selects the Buffer overload andfindDbtProjectsdoes not compile — 4 errors (TS2322/TS2367/TS2345).This is the same class of defect META-244 already fixed here once, for
@workspacejson/spec.types/ambient.d.tsstill shadowsnode:fsfor the remaining packages, and the guard has ashadowed-standard-typesrule for@workspacejson/*but nothing equivalent for Node builtins. Worth a follow-up.Boundary
@workspacejson/datahub-adapterwas never on npm and remains unpublished in its new home.